Text information aggregation with centrality attention

A lot of natural language processing problems need to encode the text sequence as a fix-length vector, which usually involves an aggregation process combining representations all words, such pooling or self-attention. However, these widely used approaches do not take higher-order relationships among words into consideration. Hence we propose new way obtaining weights, called eigen-centrality More specifically, build fully-connected graph for in sentence, then compute attention score each word. The explicit modeling is able capture some dependency helps us achieve better results 5 classification tasks and one SNLI task than baseline models pooling, self-attention, dynamic routing. Besides, order dominant eigenvector graph, adopt power method algorithm get measure. Moreover, also derive iterative approach gradient reduce both memory consumption computation requirement.